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About  the  Working  Paper  Series 


This  article  is  one  in  a  series  of  papers  addressing  one  or  more  issues  of  critical 
importance  to  the  acquisition  profession.  A  working  paper  is  a  forum  to  accomplish  a 
variety  of  objectives,  such  as:  (1 )  present  a  rough  draft  of  a  particular  piece  of 
acquisition  research,  (2)  structure  a  “white  paper”  to  present  opinion  or  reasoning,  (3) 
put  down  one’s  thoughts  in  a  “think  piece”  for  collegial  review,  (4)  present  a  preliminary 
draft  of  an  eventual  article  in  an  acquisition  periodical,  (5)  provide  a  tutorial  (such  as  a 
technical  note)  to  accompany  a  case  study,  and  (6)  develop  a  dialogue  among 
practitioners  and  researchers  that  encourages  debate  and  discussion  on  topics  of 
mutual  importance.  A  working  paper  is  generally  the  “internal”  outlet  for  academic  and 
research  institutions  to  cultivate  an  idea,  argument  or  hypothesis,  particularly  when  in  its 
infant  stages.  The  primary  intent  is  to  induce  critical  thinking  about  crucial  acquisition 
issues/problems  that  will  become  part  of  the  acquisition  professional  body  of 
knowledge. 

It  is  expected  that  articles  in  the  working  paper  series  will  eventually  be  published 
in  other  venues,  such  as  in  refereed  journals  and  other  periodicals,  as  technical  reports, 
as  chapters  in  a  book,  as  cases  or  case  studies,  as  monographs,  or  as  a  variety  of  other 
similar  publications. 

Readers  are  encouraged  to  provide  both  written  and  oral  feedback  to  working- 
paper  authors.  Through  rigorous  discussion  and  discourse,  it  is  anticipated  that 
underlying  assumptions,  concepts,  conventional  wisdom,  theories  and  principles  will  be 
challenged,  examined  and  articulated. 
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Abstract 


Acquisition  transformation  has  brought  numerous  changes  to  the  way  the 
Department  of  Defense  (DoD)  purchases  goods  and  services,  as  well  as  to  the  way  it 
manages  the  contractor’s  performance  after  contract  award.  One  of  the  most  significant 
changes  has  been  the  emphasis  on  the  use  of  commercial  goods  and  services  coupled 
with  the  use  of  performance-based  contracts  to  satisfy  mission-critical  needs. 

As  the  performance  of  contractors  has  become  more  critical  to  the  mission,  there 
has  been  a  significant  reduction  in  the  number  of  acquisition  personnel  available  to 
perform  what  traditionally  has  been  referred  to  as  “contract  administration  tasks.”  In 
addition,  with  the  increase  in  the  amount  of  services  contracts,  there  has  been  a  change 
in  the  skills  needed  to  manage  contracts  after  award. 

Contract  administration  is  currently  undergoing  a  paradigm  shift  because  of 
changes  to  what  the  DoD  buys  and  to  how  purchases  are  being  made.  The  move  to 
minimize  the  importance  of  detailed  specifications  and  to  rely  on  commercial  processes 
requires  more  management  than  administrative  skills.  The  bottom  line  is  that  new 
methods  and  skills  must  be  explored  and  developed  to  more  effectively  perform  what 
traditionally  has  been  referred  to  as  “contract  administration." 

This  study,  which  addresses  contracts  at  the  micro  level,  concludes  the 
Government  Performance  and  Results  Act  (GPRA)  of  1993  represents  a  viable 
framework  by  which  contracts  should  be  structured,  as  well  as  managed.  The  Act 
requires  managers  in  the  Executive  Branch  to  develop  strategic  plans  and  use 
performance  indicators  to  measure  output  and  to  evaluate  the  outcome  of  the  contract, 
as  well  as  the  contractor’s  performance. 


The  study  also  recommends  Kaplan  and  Norton's  Balanced  Scorecard  method  of 
performance  measurement,  together  with  Contractor  Performance  Information  (Past 
Performance  Reporting)  requirements,  can  be  adapted  to  the  management  of  contracts. 
Past  Performance  reporting  and  tracking  requirements  define — in  general  terms — the 


ACQUISITION  RESEARCH 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 
NAVAL  POSTGRADUATE  SCHOOL 


buyer’s  perspective  of  what  is  expected  from  the  contractor.  The  Balanced  Scorecard 
methodology  enables  managers  to  then  identify  the  critical  success  factors  and  critical 
measures  for  each  expatiation.  Both  parties  will  benefit  by  jointly  managing  these 
indicators  concurrently  throughout  the  life  of  the  contract,  rather  than  reacting  to  after- 
the-fact  reporting.  The  goal  of  such  requirements  is  to  add  joint  accountability  for  results 
to  the  buyer-seller  relationship. 
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Introduction 


Acquisition  transformation  has  brought  numerous  changes  to  the  way  the 
Department  of  Defense  (DoD)  purchases  goods  and  services,  as  well  as  to  the  way  it 
manages  the  contractor’s  performance  after  contract  award.  One  of  the  most  significant 
changes  has  been  the  emphasis  on  the  use  of  commercial  goods  and  services  coupled 
with  the  use  of  performance-based  contracts  to  satisfy  mission-critical  needs. 

As  the  performance  of  contractors  has  become  more  critical  to  the  mission,  there 
has  been  a  significant  reduction  in  the  number  of  acquisition  personnel  available  to 
perform  what  traditionally  has  been  referred  to  as  “contract  administration  tasks.”  In 
addition,  with  the  increase  in  the  amount  of  services  contracts,  there  has  been  a  change 
in  the  skills  needed  to  manage  contracts  after  award. 

Contract  administration  is  currently  undergoing  a  paradigm  shift  because  of 
changes  to  what  the  DoD  buys  and  to  how  purchases  are  being  made.  The  move  to 
minimize  the  importance  of  detailed  specifications  and  to  rely  on  commercial  processes 
requires  more  management  than  administrative  skills.  The  bottom  line  is  that  new 
methods  and  skills  must  be  explored  and  developed  to  more  effectively  perform  what 
traditionally  has  been  referred  to  as  “contract  administration.” 
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The  Primary  Research  Question 


The  primary  research  question  is:  can  performance  measurement  be  used  to 
manage  a  contractor’s  performance  after  contract  award?  This  question  is  raised  in 
response  to  a  widespread  concern  regarding  the  effectiveness  of  management 
techniques  used  by  the  federal  government  to  manage  contractor  performance  after 
contract  award.  In  addition,  acquisition  reform  has  placed  increased  emphasis  on  the 
use  of  commercial  products  and  services,  including  the  use  of  performance-based 
contracting  to  satisfy  mission-critical  needs.  These  factors,  coupled  with  significant 
reductions  in  size  and  skill  level  of  the  acquisition  workforce,  mean  new  methods  must 
be  explored  to  perform  what  have  traditionally  been  called  the  “contract  administration 
functions.” 

The  following  criteria  guided  this  study  and  its  answer  to  the  primary  research 
question: 

•  The  performance  measurement  process  must  be  compatible  with  the  overall 
management  planning  and  control  process  of  the  federal  government. 

•  The  performance  measurement  process  must  also  be  compatible  with  what  the 
agency  buys  and  the  way  it  buys  it. 

•  The  performance  measurement  process  must  add  value  to  the  buyer-seller 
relationship. 

•  To  be  selected,  the  performance  measurement  process  must  be  subject  to  cost  - 
benefit  trade-off  analysis. 

Scope  of  the  Research 

This  research  will  concentrate  on  the  federal  acquisition  process.  More 
specifically,  it  will  be  centered  on  the  contractor’s  performance  after  contract  award  and 
how  the  government  fulfills  its  oversight  responsibilities  after  contract  award.  It  will  not 
address  measuring  the  effectiveness  or  efficiency  of  Government  contracting 
organizations  at  the  macro  level. 
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THE  THEME 


"If  you  always  do  what  you  always 
did,  you  always  get  what  you 
always  got!" 

Congressman  Jim  Nessle* 


Dick  Army,  Army's  Axioms,  John  Wiley  &  Sons,  Inc.Hoboken,  N.J.,  2003,  p  25 1 . 

Figure  1.  Theme  1 

Foundation  and  Framework  for  How  the  Federal  Government 
Should  Do  Business  Every  Day 

The  idea  of  integrating  performance  into  the  federal  budget  process  is  not  new. 
Since  the  Hoover  Commission  proposed  “performance  budgeting”  in  1949,  there  has 
been  continued  interest  on  the  part  of  the  Legislative  Branch  in  measuring  the  results 
achieved  in  the  past  and  linking  them  to  future  budgets.  (See  Figure  1 .)  The  Planning, 
Programming,  and  Budgeting  System  (PPBS)  introduced  in  the  1960’s,  and  the  Zero- 
Based  Budgeting  System  employed  during  the  Carter  Administration  are  just  two 
examples  of  structures  which  tried  to  correlate  program  objectives  both  to  results  and  to 
the  means  of  achieving  program  goals. 2 

A  more  recent  example  of  a  governmental  attempt  to  determine  how  effective  an 
agency  is  in  managing  its  programs  is  the  Government  Performance  and  Results  Act 


1  Dick  Armey,  Armey’s  Axioms  (Hoboken,  NJ:  John  Wiley  &  Sons),  251 . 

2  James  J.  Cavanagh,  et  al.,  “The  Balanced  Scorecard  for  Managing  Procurement  Performance,” 
Contract  Management,  (February  1999):  13. 
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(GPRA)  of  1993.  GPRA  requires  departments  and  agencies  to  have  long-range 
strategic  plans,  develop  measurable  outcomes  and  to  report  results  achieved  annually 
for  each  program  in  their  budgets.  Previously,  managers  concentrated  on  administrating 
the  processes  and  reporting  how  well  they  were  functioning  rather  than  on  being 
concerned  if  the  output  satisfied  the  goals  upon  which  the  program  was  based. 3 

The  bottom  line  for  most  government  agencies  is  their  mission:  what  they  want  to 
achieve.  Under  GPRA,  what  the  agency  wants  to  achieve  is  expressed  in  its  strategic 
plans.  These  plans  also  include  performance  indicators  to  be  used  to  measure  the 
outputs  and  evaluate  the  outcomes  of  each  program.  The  primary  goals  of  GPRA  are  to 
ensure  strategic  planning  drives  budgeting  and  that  resource  decisions  reflect  specific 
strategic  priorities,  not  just  any  “firestorm”  that  occurs. 4 

David  M.  Walker,  comptroller  general  of  the  United  States,  writing  in  The  Public 
Manager,  emphasized  the  importance  of  the  management  concept  contained  in  GRPA 
to  how  supervisors  in  the  public  sector  should  manage.  He  asserts,  “GPRA  provides  the 
foundation  and  framework  for  how  the  federal  government  does  business  every  day.” 5 

The  key  to  strategic  success  under  this  act  is  an  understanding  of  the  elements 
of  the  GPRA  framework  and  their  interrelationships. 

•  The  congressional  budget  includes  a  strategic  performance  plan  that  defines 
what  major  programs  the  agency  intends  to  accomplish. 

•  Strategic  performance  plans  are  designed  to  establish  a  “bottom  line”  to  which 
each  government  program  can  be  held  accountable. 

•  Activities  are  actions  and  tasks  performed  within  the  organization  to  produce 
outputs  (goods  or  services). 


3  Thomas  G.  Kessler  and  Patricia  Kelly,  The  Business  of  Government:  Strategy,  Implementation  and 
Results,  Management  Concepts,  (Vienna,  VA:  Management  Concepts,  2000),  50. 

4  James  B.  Whittaker,  President’s  Management  Agenda,  A  Balanced  Scorecard  Approach,  (Vienna,  VA: 
Management  Concepts,  2003),  1. 

5  David  Walker,  “Managing  By  Results  to  Benefit  Our  Citizens.”  The  Public  Manager 
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•  Performance  indicators  are  particular  values  or  characteristics  that  are  used  to 
measure  outputs  and/or  outcomes.  They  include  such  factors  as  timeliness, 
quality,  product  integrity,  customer  satisfactions,  and  quantity. 

•  Output  is  the  goods  or  services  produced  by  a  program. 

•  Outcomes  are  the  things  that  occur  as  a  result  of  output  when  the  customer 
receives  and  uses  the  goods  or  services  produced.6 7  (See  Figure  2.) 


GPRA  Performance 
Accountability  Framework 


And  Local  Governments,  IBM 
For  Endowment  for  The  Business 
Of  Government,  Nov.2002. 13 


Figure  2.  The  GPRA  Performance  Accountability  Framework  7 

The  National  Partnership  for  Reinventing  Government  (National  Performance 
Review),  which  was  established  to  make  government  run  more  efficiently  and 
effectively,  continued  the  GPRA  theme.  This  was  followed  by  three  other  pieces  of 
legislation  that  required  strategic  plans  to  describe  how  each  government  agency  will 
deliver  quality  results  and  then  measure  its  performance  in  terms  of  output  and 
outcomes.  These  pieces  of  legislation  were  the  Federal  Acquisition  Streamlining  Act  of 


6  Kessler  and  Kelly,  56-61. 

7  Lawrence  L.  Martin,  Making  Performance-Based  Contracting  Perform:  What  the  Federal  Government 
Can  Learn  from  State  and  Local  Governments,  New  Ways  to  Manage  Series  (Arlington,  V A:  IBM  Center 
for  Endowment  for  the  Business  of  Government,  Nov.  2002),  13. 


NPS 


yUAtSTANTlA  rE R  i 


ACQUISITION  RESEARCH 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 
NAVAL  POSTGRADUATE  SCHOOL 


-5- 


1994,  the  Government  Management  Reform  Act  of  1994  and  the  Information 
Technology  Management  Reform  Act  of  1996  (Clinger-Cohen  Act). 

Some  agencies  and  departments  have  also  adapted  Kaplan  and  Norton’s 
balanced  scorecard  approach  and  are  using  balanced  performance  measures  to 
evaluate  performance  of  their  organization  as  a  whole.  As  the  name  implies,  the 
balanced  scorecard  does  not  concentrate  on  just  measuring  financial  performance.  It  is 
multidimensional  and  includes  the  customer’s  perspective,  internal  processes’ 
perspective,  and  learning  and  growth  as  measurements  of  the  financial  perspective  in 
order  to  obtain  some  balance  of  these  categories  in  relation  to  mission  achievement. 
This  approach  allows  managers  to  consider  whether  improvements  in  one  area  are 
being  achieved  at  the  expense  of  another.  Examples  of  the  use  of  the  balanced 
scorecard  by  public  organizations  include  Naval  Undersea  Warfare  Center,  FAA 
Logistics  Center,  Internal  Revenue  Service,  and  the  Department  of  Transportation. 8 

The  Federal  Acquisition  Regulations  Guide  to  Acquisition  of 
Goods  and  Services 

It  is  the  Federal  Acquisition  Regulations  (FAR)  that  provides  the  foundation  to 
guide  the  acquisition  of  goods  and  services  procured  by  the  Executive  Branch.  The 
FAR  includes  a  strategic  planning  perspective  in  the  form  of  a  “primary  vision”  for  the 
federal  acquisition  system.  The  vision  statement  dictates:  the  primary  goal  of  the 
acquisition  system  is  to  “deliver  on  a  timely  basis  a  best  value  product  or  service  to  the 
customer,  while  maintaining  the  public  trust  and  fulfilling  public  policy  objectives.”  (See 
Figure  3.) 

The  FAR  goes  on  to  enumerate  four  major  critical  success  factors  by  which  the 
process  should  be  evaluated.  These  are, 

•  Customer  satisfaction  of  cost,  quality,  and  timeliness 


8  Whittaker,  249-257. 
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Minimization  of  administrative  cost 


•  Maintenance  of  the  public  trust,  and 

•  Fulfillment  of  public  policy  objectives. 


Acquisition  Vision  and 
Performance  Standards 


"TO  DELIVER  ON  A  TIMELY  BASIS  THE  BEST 
VALUE  PRODUCT  OR  SERVICE  TO  THE  CUSTOMER 
WHILE  MAINTAINING  PUBLIC  TRUST  AND 
FULFILLING  POLICY  OBJECTIVES." 


FAR  1.102 


Figure  3.  Acquisition  Vision  and  Performance  Standards 


Each  of  these  critical  success  factors  lends  itself  to  the  establishment  of 
performance  indicators  which  measure  the  output  and  outcome  as  they  are  presented  in 
the  GPRA  framework. 

For  defense  weapons  systems,  Department  of  Defense  5000  (DoD  5000)  series 
directives  govern  all  acquisition  programs.  Important  to  the  management  of  each 
defense  program  is  the  Acquisition  Program  Baseline  (APB).  The  APB  serves  as  a 
critical  success  indicator  because  the  baseline  also  represents  desired  results  in 
measurable  terms.  Every  Program  Manager  (PM)  is  required  to: 
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establish  program  goals  for  a  minimum  number  of  cost,  schedule  and 
performance  parameters  that  describe  the  program  over  its  life  cycle.  Approved 
program  baseline  parameters  shall  serve  as  control  objectives.  PM  shall  identify 
deviation  from  approved  program  baseline  parameters  and  exit  criteria.10 

In  addition  to  ABP  data,  DoD  Instruction  5000.2  contains  references  to  fifty  pieces  of 
information  required  by  the  statutes  or  regulations  that  may  be  applicable  during  product 
development  milestones  and  phases.  This  regulatory  program  information  is  to  be 
tailored  to  fit  the  particular  conditions  of  each  individual  program.  This  program  includes 
a  Contractor  Cost  Data  Report,  which  is  required  for  non-commercial  contracts  not 
procured  under  competitive  procedures  that  include  information  on  major  contracts  and 
subcontracts  for  Acquisition  Category  I  Programs  over  $50  million  and  high-risk  or  high- 
technical  interest  contracts  between  $7  and  $50  M.11 

Changing  What  We  Buy  and  the  Way  We  Buy  It 

In  FY  2002,  the  Federal  Government  continued  its  long-standing  record  as  the 
largest  buyer  of  goods  and  services  in  the  world.  In  that  year,  contract  actions 
exceeded  $250.2  billion.  Department  of  Defense  (DoD)  contracts  represented  66%  of 
that  total — e.g.,  $164.7  billion. 

In  the  last  ten  years,  there  have  been  significant  increases  in  the  quantity  of 
commercial  items  purchased  by  the  DoD.  Primary  reasons  for  this  were:  (a)  the  fact  that 
more  and  more  innovative  products  are  available  commercially-off-the-shelf  (COTS) 
that  would  satisfy  government  requirements;  (b)  Federal  Acquisition  Streamlining  Act 
(FASA)  has  simplified  procedures  for  procurement  of  low-cost,  low-risk  commercial 
supplies  and  services;  and  (c)  military  specifications  and  standards  are  replaced  by 
performance  standards  or  commercial  standards  when  possible.  Consequently,  FAR 


9  Federal  Acquisition  Regulation  (FAR)  1.102 

10  Defense  Acquisition  System,  DoD  Directive  5000.1, 4.3.4  Discipline,  May  12,  2003. 

11  Statutory,  Regulatory,  and  Contract  Reporting  Information  and  Milestone  Requirements,  May  12,  2003, 
Enclosure  3,  DoD  Instruction  5000.2. 
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policy  now  requires  the  contracting  officer  of  each  acquisition  to  initially  look  to  the 
commercial  market  as  a  source  for  goods  and  services.  12 

The  dollar  value  of  purchases  for  services  now  exceeds  what  is  spent  for 
supplies  and  equipment.  In  2003,  in  particular,  the  General  Accounting  Office  (GAO) 
stated  during  the  last  five  years  DoD  spent  more  to  procure  services  than  it  did  for 
supplies  and  equipment.  In  fact,  approximately  50%  of  DoD’s  spending  in  2002  was  for 
services.  Between  1999  and  2002,  spending  for  services  increased  approximately  19% 
to  $93  billion.13  During  this  same  period,  the  number  of  contractors  grew  by  over 
727,000,  while  the  civil  service  workforce  decreased.14 

Part  of  this  trend  has  been  due  to  the  fact  that  over  the  past  few  years,  policy  has 
emphasized  consideration  of  private-sector  firms  as  alternative  service  providers  of 
tasks  that  can  be  classified  as  being  inherently  commercial.  The  Federal  Activities 
Inventory  Reform  Act  (FAIRA)  calls  for  the  head  of  each  agency  to  decide  which  activity 
would  be  subject  to  competition  under  the  provisions  of  OMB  Circular  A-76. 15 

Two  examples  of  activities  that  have  been  contracted  out  to  the  private  sector 
have  been  facility  maintenance  and  personnel  support  services  at  military  installations 
and  bases.  Information  technology  is  another  area  considered  to  be  commercial  in 
nature  and,  therefore,  subject  to  competitive  sourcing. 


12  Federal  Acquisition  Regulation  (FAR)  10.002 

13  Contract  Management:  High-Level  Attention  Needed  to  Transform  DoD  Services  Acquisition,  GAO 
Report.  September  03,  GAO-03-935. 

14  Jason  Peckenpaugh,  “Contractor  Workload  Grows  as  Civil  Service  Shrinks,”  Government  Executive 
Magazine  (September  4,  2003).  In  February  2004,  the  Office  of  Management  and  Budget  (OMB)  revised 
A-76  to  allow  federal  agencies  to  compete  for  this  work  against  non-federal  organizations.  The 
competition  determines  which  service  provider  can  supply  mission-related  support  and  meet  business 
requirements  at  the  best  price  without  compromise  to  quality  or  performance.  In  its  April  12,  2004  edition, 
the  Federal  Times  reported  between  1998  and  2003  DoD  civilian  and  acquisition  personnel  declined 
7.5%;  likewise,  civilian  staffing  of  the  Defense  Contract  Management  Agency  (DCMA)  fell  23.6% 

15  Federal  Acquisition  Council,  “Manager’s  Guide  to  Competitive  Sourcing,”  Second  Edition,  February  20, 
2004.  Competitive  sourcing  has  been  described  as  a  “tool  to  help  you  benchmark  your  organizations 
against  other  possible  service  providers,  to  stimulate  your  organization  to  think  of  ways  to  change  in  order 
to  become  the  best  it  can  be.”  And,  “It  involves  conducting  public-private  reviews  or  competitions  to 
assess  how  best  to  deliver  services  to  the  public.” 
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There  have  been  other  changes  to  what  the  federal  government  buys  and  how  it 
purchases  those  goods  and  services.  Beginning  in  1995,  acquisition  managers  have 
been  encouraged  to  think  more  creatively  and  look  for  innovative  ways  to  procure 
supplies  and  services.  This  paradigm  shift  was  propelled  by  two  significant  pieces  of 
legislation  which  have  been  discussed  previously,  FASA  and  the  FARA.  Both  urged 
acquisition  managers  to  look  for  alternative  ways  of  doing  business  that  was  cheaper, 
better,  and  faster — as  long  as  it  made  good  business  sense.  The  only  constraint  was 
that  the  FAR,  Executive  Order  or  agency  regulations  could  not  prohibit  the  particular 
course  of  action.16 

Examples  of  the  some  of  the  more  significant  changes  are  listed  below.  Each 
was  brought  about  by  a  policy  that  tolerated  thinking-outside-the-box.  (See  Table  1 .) 

1 .  Use  of  Government  Purchase  Cards:  Government-wide  policy  has  made  use  of 
the  Purchase  Cards  as  the  preferred  method  of  procuring  commercial  items 
below  the  micro-purchase  threshold  of  $2,500  ($2,000  for  construction).  In  FY 
2003,  purchase  card  usage  history  showed  there  were  10.7  million  purchase 
card  transactions  totaling  $7.2  billion  in  sales;  these  sales  thereby  eliminated  a 
significant  workload  which  previously  had  been  performed  by  contracting 
personnel  and  the  management  of  relatively  small-value  purchases. 

2.  Use  of  e-Commerce:  The  DoD  has  further  facilitated  the  purchase  of  consumable 
supplies  by  establishing  the  DoD  EMALL.  The  DoD  EMALL’s  objective  is  to  be 
the  single  electronic  point-of-entry  for  purchasers  to  locate  and  acquire  off-the- 
shelf,  finished  goods  and  services  from  the  commercial  market  place  as  well  as 
from  government  sources.  It  creates  a  paperless  environment,  thereby 
simplifying  the  contracting  job. 

3.  Expanded  Use  of  Federal  Supply  Schedules:  There  has  been  a  significant 
increase  in  the  use  of  indefinite-delivery,  indefinite-quantity  (IDIQ)  contracts.  Use 
of  a  Federal  Supply  Schedule  permits  contracting  officers  to  initiate  individual 
orders  for  supplies  or  services  from  commercial  firms  using  IDIQ  contracts 
previously  established  by  the  General  Services  Administration  (GSA).  This 
system  was  not  previously  permitted  because  the  source  of  contracting  authority 


16  Federal  Acquisition  Regulation  (FAR)  1.1 02d 
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was  outside  GSA.  Under  this  method,  however,  the  ordering  agency  is 
responsible  for  determining  if  contract  performance  is  acceptable  and  for  taking 
the  necessary  action  if  it  is  not.  This  method,  formerly  known  as  the  Federal 
Supply  Schedule  Program,  accounted  for  contracts  which  totaled  7  billion  dollars 
in  FY  2002  alone.  It  has  not  only  simplified  the  contracting  officers’  jobs  by 
eliminating  the  need  to  participate  in  the  source-selection  process,  but  it  has  also 
greatly  reduced  procurement  lead-times. 

4.  Strategic  Purchasing  of  Commodities:  Use  of  Cross-functional  Sourcing  Teams’ 
Leverage  in  Terms  of  Buying  Power  by  Strategic  Source  Centralized  Purchasing: 
The  services  are  now  looking  to  optimize  their  buying  power  by  creating 
“Commodity  Councils.”  The  objective  is  to  develop  centralized  purchasing 
strategies  for  different  commodities.  IBM,  Daimler/Chrysler,  and  Cessna  have 
successfully  used  it.  Adoption  of  this  concept  means  the  service  will  enter  into 
enterprise-wide  larger  volume  contracts  with  fewer  suppliers  rather  than  having 
each  user  contract  for  its  own  needs.  This  method  of  acguisition  gives  the  buyer 
the  advantages  of  leverage.  It  also  eliminates  a  significant  amount  of 
administrative  workload  at  multiple  installations.  17 

5.  Use  of  Performance-Based  Contracting  Methods:  Performance-based  contracts 
are  structured  around  terms  that  describe  measurable  output,  guality  or 
outcomes  in  the  form  of  measurable  standards  for  the  work  to  be  performed, 
rather  than  broad  or  precise  statements  of  work  regarding  the  way  services  are 
to  be  performed.  This  method  of  procurement  emphasizes  telling  the  provider 
what  is  wanted  rather  than  how.  It  also  encourages  contractor  rewards  for 
exceptional  performance  or  fee  reduction  when  performance  is  not  satisfactory. 

It  is  DoD  policy  that  50%  of  all  service  contracts  be  performance-based  by  2005. 
Experience  has  shown  that  although  additional  work  is  reguired  with  this  method, 
initially  describing  tasks  to  be  performed  and  guantifying  desired  outputs  and 
outcomes  greatly  reduces  the  contracting  officers’  involvement  after  contract 
award. 


17  For  more  on  this  subject  see  David  Burt,  Donald  Dobler,  and  Stephen  L.  Starling,  World  Class  Supply 
Management SM:  The  Key  to  Supply  Chain  Management,  Seventh  Edition  (New  York:  McGraw-Hili,  2003), 
328. 
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Results  of 

Out-Of-The-Box  Thinking 

•  Purchase  Card  Used  for  Micro  Purchases 

-  Purchases  <  $2,500 

-  Reduced  Role  of  Contracting  Officer 

•  Use  Of  E-Commerce 

-  Single  Point  of  Entry  to  Commercial  Market 

•  Expanded  Use  of  Federal  Supply  Schedules 

-  Using  of  Previously  Established  Contracts 

•  Strategic  Purchasing  by  Commodities 

-  Leverage  Buying  Power 

•  Use  of  Performance  Based  Contracting  Methods 

-  50%  of  Services  Contracts  to  be  PBSC  by  2005 

Table  1.  Results  of  Thinking-Out-Of-The-Box 

As  government  acquires  more  and  more  services  via  contract,  the  lead-time  from 
acquisition  decision  to  mission  impact  has  become  less  and  less.  Regarding  a  weapon 
system,  years  can  pass  before  the  mission  impact  of  a  poor  decision  made  during 
design,  production,  or  provisioning  of  spare  parts  can  be  discovered.  In  contrast,  failure 
to  perform  a  mission-critical  service  can  have  an  almost  instantaneous  negative  impact. 

This  reality  requires  a  re-examination  of  the  relationships  between  the  buyer  and 
seller.  Policies  initiated  during  acquisition  reform  have  made  it  possible  to  move  away 
from  an  arms-length  buyer-seller  relationship  in  favor  of  greater  communication  and 
even  inclusion  of  the  contractor  as  a  member  of  the  Acquisition  Team.  18 

Contractor  Oversight  after  Contract  Award 

Reform  of  federal  procurement  policy  has  resulted  in  increased  flexibility  for 
contracting  officers,  program  managers,  and  other  acquisition  personnel.  Changing 
times  call  for  innovative  ways  to  achieve  results.  As  noted  above,  this  innovation 
occurred  primarily  in  the  methods  by  which  supplies  and  services  are  purchased.  In 


18  Federal  Acquisition  Regulation  (FAR)  1 . 1 02  (c)  “The  Acquisition  T earn  consists  of  all  participants  [. . .] 
including  technical,  supply,  and  procurement  communities,  but  also  the  customers  they  serve,  and  the 
contractors  who  provide  the  products  and  services.” 
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relation  to  contracts,  the  impact  of  acquisition  reform  on  the  function  of  contract 
administration  has  not  been  fully  recognized.  19 

After  contract  award,  oversight  of  the  contractor’s  performance  has  traditionally 
been  referred  to  as  “contract  administration.”  The  objectives  of  contract  administration  is 
to  gain  assurance  that  a  quality  product  will  be  delivered  to  the  ultimate  user  in 
accordance  with  cost  and  schedule  projections,  as  well  as  to  retain  the  confidence  of 
the  tax  payer  in  proving  his/her  tax  dollars  are  being  spent  wisely. 

Oversight  includes  the  employment  of  government  personnel  for  such  tasks  as 
monitoring  contractor  performance,  inspecting  and  accepting  delivered  products  or 
services,  modifying  the  contract,  resolving  technical  and  delivery  problems,  reviewing 
contractor’s  processes,  auditing  contractor  systems,  and  applying  rewards  and 
penalties  when  needed. 20 

Professor  Steve  Kelman,  former  head  of  the  Office  of  Federal  Procurement 
Policy  (OFPP),  and  now  a  Professor  of  Public  Management  at  John.  F.  Kennedy  School 
of  Government,  believes  contracting  can  no  longer  be  considered  primarily  an 
administrative  function.  His  thesis  is  supported  by  the  fact  that  a  high  percentage  of  an 
agency’s  budget  is  spent  on  contracts  for  supplies  and  services.  He  recommends 
contracting  management  be  considered  a  core  competence  within  each  federal 
organization.  He  believes  critical  responsibilities  that  accompany  the  job  include 
performance  measurement  and  management  of  the  contractor. 21 


19  Wendell  C.  Lawther,  Contracting  in  the  21st  Century:  A  Partnership  Model  (Arlington,  VA: 
Pricewaterhouse-Coopers  Endowment  for  the  Business  of  Government,  2002),  6.  Professor  Lawther ‘s 
thesis  is:  the  reform  of  federal  procurement  policy  has  resulted  in  increased  flexibility  for  contracting 
officers,  program  managers,  and  related  procurement  personnel.  The  impact  of  these  reforms  on  the 
function  of  contract  administration  has  not  been  fully  recognized. 

20  Federal  Acquisition  Regulation  (FAR)  42.302  list  65  “normal  contract  administration  functions”  which 
Administrative  Contracting  Officers  have  the  authority  to  automatically  carry  out. 

21  Seven  Kelman,  “Remaking  Federal  Procurement:  Visions  of  Governance  in  the  21st  Century,”  Working 
Paper  No.  3,  May  2002.,  2.  Professor  Kelman  supports  this  point  by  the  fact  the  DoD,  Department  of 
Energy  (DoE),  and  the  National  Aeronautics  and  Space  Administration  (NASA)  spend  a  significant 
amount  of  their  budgets  on  contracted  products  and  services-46%,  94%  and  78%  respectively. 
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The  nature  and  extent  of  contract  administration  varies  from  contract  to  contract. 
It  can  vary  from  a  minimum  level  of  involvement  in  the  contractor’s  activities  to  a 
maximum  level  of  almost  dictating  the  methods  by  which  supplies  and  services  are 
purchased.  This  is  in  contrast  to  the  impact  of  total  engagement.22  The  depth  of 
oversight  has  traditionally  taken  one  of  the  following  approaches.  (See  Figure  4.) 

•  Minimum  oversight  is  associated  with  small-scale  jobs,  projects  where  the 
management  and  control  is  essentially  left  to  the  discretion  of  the  supplier  in  the 
purchase  of  commercial  items.  The  contractor  is  most  often  permitted  to  follow 
its  own  production,  assembly,  testing  and  quality  process/procedures.  For 
commercial  items,  inspection  and  testing  by  the  government  is  completed  in  a 
manner  consistent  with  accepted  industry  practices. 

•  The  intermediate  level  of  oversight  occurs  when  the  procurement  is  large  in 
scale,  or  the  supplies  and  equipment  being  procured  are  unique  to  the 
government.  The  buyer  sometimes  imposes  extensive  data  collection  and  status 
reporting  on  the  seller.  However,  in  this  situation,  the  seller’s  managing  systems 
and  techniques  generally  provide  the  buyer  with  the  necessary  performance- 
status  information.  When  government  unique  standards  are  required,  the 
contractor  can  also  be  required  to  follow  government  production,  assembly  and 
testing  processes  as  a  term  and  condition  of  the  contract. 

•  The  most  intensive  level  involves  large  projects,  research  and  development 
efforts  or  the  acquisition  of  major  systems.  In  this  case,  the  supplier’s  internal 
management  systems  and  processes  may  not  be  fully  acceptable.  Also  in  this 
case,  the  buyer  often  imposes  process  specifications  or  standards  of 
performance  upon  the  supplier.  Often,  the  contractor  is  required  to  follow 
government  production,  assembly  and  testing  processes  as  a  term  and  condition 
of  the  contract. 


22  John  Cibinic,  Jr.  and  Ralph  C.  Nash,  Jr.  Administration  of  Government  Contracts,  Second  Edition, 
Government  Contracts  program,  (Washington,  D.C.:  George  Washington  University,  1986),  2. 
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Level  of  Contractor  Oversight  by 
The  Government 


Figure  4.  Level  of  Contractor  Oversight  by  Government 

An  important  responsibility  of  contracting  personnel  after  contract  award  is 
evaluating  a  contractor’s  performance  information  on  designated  contracts  on  an  annual 
basis,  as  well  as  at  the  time  of  completion  of  the  contract.23  The  FAR  then  requires  the 
source-selection  authority  to  reconsider  this  record  of  past  performance  for  future 
source-selection  purposes.  Knowledge  of  a  contractor’s  record  of  contract  performance 
is  seen  as  a  way  to  mitigate  the  risk  of  selecting  a  contractor  with  a  poor  track  record. 

Contractor  performance  reports  are  collected  and  maintained  in  a  web-enabled 
library  developed  by  Naval  Sea  Logistics  Center  for  the  DoD,  commonly  referred  to  as 
CPARS  (Contractor  Perform  Assessment  Reporting  System).  The  type  of  information 
reported  includes: 


23  DoD  Deviation  to  FAR  42.1502  requires  evaluations  of  contractor  performance  for  each  contract  in 
excess  of:  (a)  $5,000,000  for  systems  and  operational  support,  (b)  $1 ,000,000  for  services,  information 
technology  contracts,  and  (c)  $1 00,000  for  fuels  and  health  care  contracts. 
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•  Contractor’s  record  of  conforming  to  contract  requirements  and  standards  of 
good  workmanship, 

•  Contractor’s  record  of  forecasting  and  controlling  cost, 

•  Contractor’s  adherence  to  contract  schedules,  including  the  administrative 
aspects  of  performance  contractor’s  history  of  reasonable  and  cooperative 
behavior  and  commitment  to  customer  satisfaction,  and 

•  Contractor’s  business-like  concern  for  the  interests  of  the  customer. 

The  study  concludes  that  both  parties  would  jointly  benefit  by  managing  these  indicators 
concurrently  throughout  the  life  of  the  contract,  rather  than  reporting  on  them  after-the- 
fact. 

The  Cost  of  Oversight 

Not  gone  unnoticed  has  been  the  premium  paid  by  the  government  for  the 
oversight  methods  and  procedures  used  by  the  buyer  to  assure  the  timely  delivery  of 
the  needed  supplies,  services  and  the  confidence  of  the  taxpayer.  Between  1986  and 
1992,  at  least  six  studies  were  made  of  the  regulatory/oversight  cost  premium  being 
passed  on  to  the  government  buyer  by  contractors.  Rough  Order  of  Magnitude  (ROM) 
estimates  of  the  added  cost  premium  cited  in  these  reports  ranged  from  5  to  50  percent 
in  potential  cost  reductions.  The  potential  cost  avoidance  cited  by  all  of  these  studies 
no  doubt  had  an  influence  on  FASA  and  other  DoD  acquisition  policies  before  and  after 
the  Acquisition  Streamlining  Act  in  1994.  A  more  recent  assessment  of  the  DoD 
regulation  and  oversight  burden,  published  by  RAND  in  2001,  estimated  the  oversight 
burden  to  range  from  1  to  6  percent. 24 


24  Mark  Lorell,  and  John  C.  Graser,  An  Overview  of  Acquisition  reform  Savings  Estimates,  (MR-1329-AF) 
Project  Air  Force  (Santa  Monica,  CA:  RAND,  2001),  117-118.  (Studies  were  performed  by  Honeywell  in 
1 986of  contractor  cost  for  20  programs,  RAND  in  1 988  of  regulatory  costs,  Office  of  Technology 
Assessment  in  1989  of  total  DoD  acquisition  budget,  Center  for  Strategic  and  International  Studies  in 
1991  of  cost  premium  on  identical  items,  Carnegie  Commission  in  1992  of  total  DoD  acquisition  budget; 
and  American  Defense  Preparedness  Association  in  1992  of  product  cost.) 
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However,  there  has  been  recent  criticism  regarding  how  well  the  DoD  oversees 
contractor  performance.  A  recent  example  was  a  series  of  articles  published  in  the 
Federal  Times  alleging  that  the  number  of  auditors  and  inspectors  had  been  cut  back 
too  far  and  the  consequences  were  that  the  government  was  “being  ripped-off.” 25 

In  response  to  this  criticism,  Professor  Kelman  pointed  out  oversight  is  not 
costless... that  fixating  on  traditional  oversight  methods  while  ignoring  other  critical 
contract  management  tasks  (such  as  establishing  performance  metrics,  inciting 
contractor  technical  experts  and  government  subject-matter  experts  to  share 
information,  and  motivating  both  government  and  contractor  personnel  to  achieve 
results)  is  not  the  answer.26 

What  is  Performance  Measurement? 

Performance  is  defined  in  the  Random  House  Webster’s  College  Dictionary  as: 
“The  execution  or  accomplishment  of  work,  acts,  feats,  etc.  [. . .]  The  act  of  performing. 
[It  is]  the  manner  in  which,  or  efficiency  with  which  something  reacts  or  fulfills  its 
intended  purpose.” 27 

Professors  Lebas  and  Euske  state  performance  is  continuous  and  has  a  cause 
and  effect.  Performance  exists  only  if  outcome  and  results  can  be  described  or 
measured  so  as  to  be  accurately  communicated;  only  in  this  way  can  someone  decide 
to  do  something  within  the  shared  model  of  causal  relationships.  Performance  does  not 
have  the  same  meaning  if  the  evaluator  is  inside,  rather  than  outside  the  organization. 
Use  of  the  term  performance  should  be  reserved  for  all  processes  that  lead  to  a 
potential  or  future  sequence  of  outcomes  and  results.28 


25  “Blind  Trust:  Who  is  watching  DoD  contractors?”  Federal  Times,  (April  5,  12  and  19,  2004).. 

26  Steve  Kelman,  “Oversight  is  not  the  only  way  to  judge  a  procurement  system,”  Federal  Times.  (May  31 
2004). 

27  Random  Flouse  Webster’s  College  Dictionary,  (New  York:  Random  House,  1994)”  1003. 

28  Lebas  and  Kenneth  Euske,  “Theoretical  and  conceptual  issues,”  Business  Performance  Measurement: 
Theory  and  Practice,  ed.  A.D.  Neely  (Cambridge,  U.K.:  Cambridge  University  Press,  2002). 
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The  Government  Accounting  Standards  Board  views  performance  as  a 
multidimensional  concept  consisting  of  outputs,  quality  and  results.29 

Random  House  Webster’s  College  Dictionary  defines  measurement  as  “the  act 
of  measuring  [...]  a  dimension,  extent,  range,  etc.  or  ascertained  by  measuring.”30 

In  its  simplest  terms,  performance  measurement  is  the  comparison  of  actual 
levels  of  performance  to  pre-established  target  levels  of  performance. 31  (In  the 
terminology  of  GPRA,  this  is  comparing  output  to  pre-established  outcome  objectives.) 

Traditionally,  performance  measurement  has  been  a  key  element  of  what 
management  gurus  have  referred  to  as  the  “control  function.”  The  objectives  of 
performance  measurement  are: 

assessing  progress  toward  achieving  predetermined  goals,  including  information 
on  the  efficiency  with  which  resources  are  transformed  into  goods  and  services 
(outputs),  quality  of  those  outputs  (how  well  they  are  delivered  to  clients  and  the 
extent  to  which  clients  are  satisfied)  and  outcomes  (the  results  of  a  program 
activity  compared  to  its  intended  purpose)  and  the  effectiveness  of  government 
operations  in  terms  of  their  specific  contributions  to  program  objectives. 32 

In  their  book  The  Business  of  Government:  Strategy,  Implementation  and 
Results,  Thomas  Kessler  and  Patricia  Kelly  recommend  performance  measures  must 
be  placed  in  any  plan  immediately  after  the  statements  of  goals.  This  strategic  location 
forces  the  planner  to  think  critically  to  define  applicable  performance  measures  and  tie 


29  Lawrence  L.  Martin,  “Making  Performance-Based  Contracting  Perform:  What  the  Federal  Government 
Can  Learn  from  State  and  Local  Governments,”  New  Ways  to  Manage  Series  (Arlington,  VA:  IBM 
Endowment  for  the  Business  of  Government,  November  2002),  13. 

30  Random  House  Webster’s  College  Dictionary  (New  York:  Random  House,  1994),  840. 

31  “Establishing  and  Maintaining  a  Performance-Based  Management  Program,”  The  Performance-Based 
Management  Handbook,  Volume  One,  Performance-Based  Management  Special  Interest  Group  (PBM 
SIG),  September  2001 .  http:///www. orau.gov/pbm. 

32  James  B.  Whittaker,  President’s  Management  Agenda,  A  Balanced  Scorecard  Approach  (Vienna,  VA: 
Management  Concepts,  2003),  259. 
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them  to  each  goal  early  in  the  planning  process.  The  authors  go  further  by  stating,  in 
order  to  be  effective,  a  performance  measure  must  include  the  following  elements  (See 
Table  2): 

•  Performance  measures:  Characteristics  relative  to  the  overall  strategy,  such  as 
cost,  cycle  time,  quality,  and  quantity. 

•  Performance  indicators:  Unit  of  measure  as  dollars,  hours,  errors  per  1000, 
number  of  reports  each  period  that  can  be  used  to  measure  output  and  outcome. 

•  Baseline:  An  initial  starting  measurement. 

•  Performance  target:  Desired  level  of  performance  expressed  as  a  tangible, 
measurable  objective  against  which  achievement  can  be  compared  relative  to  a 
baseline  measure. 

•  Tolerances:  Acceptable  ranges  of  variance  from  performance  targets.  33 


Elements  of 

Performance  Measurement 


•  Performance  Measure: 

-  A  Measured  Characteristic/s  Relative  to  Overall 
Strategy 

•  Performance  Indicators 

-  Evaluation  of  Output  or  Outcome 

•  The  baseline 

-  Initial  Starting  Measurement 

•  Desired  Level  of  Performance 

-  Compared  With  Baseline 

•  Tolerances 

-  Acceptable  Range  of  Variances  from  Performance 
Target 

Thomas  G.  Kessler,  Patricia  Kelly,  The  Business  of  Government:  Strategy,  Implementation <5  Results, 
Management  Concepts,  Vienna,  \M.  2000,  86. 


Table  2.  Elements  of  a  Performance  Measure 


33  Thomas  G.  Kessler  and  Patricia  Kelly,  The  business  of  Government:  Strategy,  Implementation  and 
Results  (Vienna,  VA:  Management  Concepts,  2000),  86. 
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As  anyone  in  public  or  private  industry  can  attest,  performance  measurement 
does  not  stop  when  variances  from  planned  performance  have  been  identified. 
Leading-edge  organizations  seek  to  create  an  efficient  and  effective  performance- 
management  system.  James  Whitaker,  author  of  President’s  Management  Agenda,  A 
Balanced  Scorecard  Approach,  dictates  managers  must  also: 

use  performance  measurement  information  to  effect  positive  change  in 
organizational  culture,  systems,  and  processes  by  helping  to  set  agreed-upon 
performance  goals,  allocating  and  prioritizing  resources,  informing  managers  to 
either  confirm  or  change  current  policy  or  program  directions  to  meet  those 
goals,  and  sharing  results  of  performance  in  pursuing  those  goals. 

These  actions  are  part  of  the  performance  management  system. 34 

Professor  Marshall  Meyer  of  the  University  of  Pennsylvania’s  Wharton  School 
provides  a  schematic  of  how  performance  measures  are  used  (See  Figure  5).  He 
contends  performance  measurement  fulfills  seven  purposes.  In  addition  to  serving  as  a 
control  metric,  which  he  refers  to  as  “Look  back,”  performance  metrics  are  also  used  to 
“Compensate,”  “Motivate”  and  “Look  Ahead”  to  assess  the  future.  Use  of  performance 
measures  becomes  even  more  significant  as  an  organization  increases  in  size.  In 
larger  organizations  (the  area  within  the  triangle),  metrics  are  reported  up  in  the  chain  of 
command  (“Roll[ed]  up”)  so  management  can  determine  if  the  desired  results  have 
been  achieved  and  can  adjust  goals  or  establish  future  goals  accordingly.  These  are 
then  “Roll[ed]  Back”  down  the  organization  in  the  form  of  financial  and  non-financial 
objectives.  Another  value  performance  measurement  this  model  provides  is  that  it  can 


34  Whittaker,  260. 


ACQUISITION  RESEARCH 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 
NAVAL  POSTGRADUATE  SCHOOL 


-20- 


also  be  used  to  “Compare”  to  other  organizations  across  functional  boundaries 
horizontally  or  to  compare  to  other  business  units. 35 


The  Seven  Purposes 
of  Performance  Measures 


Marshall  Meyer,  Rethinking  Performance  Measurement,  Cambridge  University  Press,  UK,  2002,  page  31 


Figure  5.  The  Seven  Purposes  of  Performance  Measures 

Some  Examples  of  How  DoD  Now  Uses  Performance 
Measurement  to  Manage  Contracts  After  Award 

Award  Fee  Type  of  Pricing  Arrangements:  This  type  of  contract  provides  a 
monetary  base  fee,  plus  an  additional  fee  that  is  awarded  periodically  based  on  a 
subjective  evaluation  by  the  acquisition  team  of  how  well  the  contractor  satisfied 
predetermined  evaluation  factors.  Contractor’s  performance  in  such  areas  as 
responsiveness,  quality,  timeliness,  management  of  risk  and  cost  contract  are  examples 
of  possible  evaluation  factors.  Important  to  note,  however,  is  the  acquisition  team  can 
change  its  evaluation  criteria  at  designated  milestones  during  contract  performance. 


35  Marshall  W.  Meyer,  Rethinking  Performance  Measurement:  Beyond  the  Balanced  Scorecard, 
(Cambridge,  U.K.:  Cambridge  University  Press,  2002),  30. 
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A  good  example  of  how  performance  measurement  can  be  used  to  evaluate 
contractor  effectiveness  can  be  found  in  the  contractor-operated  Arnold  Engineering 
Development  Center  (AEDC)  USAF.  AEDC,  whose  mission  is  to  support  new 
aerospace  systems  by  conducting  R&D  testing  and  to  evaluate  for  the  DoD  and  other 
government  and  commercial  agencies,  has  been  using  competitively-awarded  cost- 
plus-award  fee-type  pricing  arrangements  for  over  25  years. 

Temple  Bowling  IV,  Senior  Contracting  Official  at  AEDC,  feels  the  award  fee 
feature  has  enabled  the  Air  Force  to  use  the  profit  motive  to  incentivize  contractors  to 
continually  improve  productivity  and  quality  while  controlling  cost.  On  the  current 
contract,  in  over  eight  award  fee  evaluation  periods,  AEDC  tracked  an  average  of  283 
separate  performance  metrics.  It  should  be  noted  that  the  importance  the  government 
placed  on  performance  factors  to  be  measured  did  not  remain  static.  Between 
evaluation  periods,  on  average  22%  of  the  performance  goals  were  adjusted;  28%  of 
the  metrics  from  the  previous  period  were  deleted;  and  an  average  of  18%  new 
performance  measures  were  added. 36 

An  advantage  of  this  type  of  contract  is  that  it  improves  communication  between 
the  two  parties.  A  disadvantage  to  both  parties  is  the  fact  that  there  is  a  substantial 
administrative  burden  associated  with  the  evaluation  and  review  of  the  award  fee 
evaluation  information. 

Program  Baseline  Parameters:  DoD  5000  series  directives,  which  provide 
management  principles  and  mandatory  policies  and  procedures  for  managing  all 
acquisition  programs,  require  every  program  manager  to  establish  goals  for  the 
minimum  number  of  cost,  schedule  and  performance  parameters  that  describe  the 
program  over  its  life  cycle. 37 


36  Data  provided  by  Mr.  Temple  Bowling  IV,  Senior  Center  Contracting  Official,  AEDC/PK,  February 
2004. 

37  DoD  Directive  5000.1,  “The  Defense  Acquisition  System,”  May  12,  2003,  Paragraph  4.3.4. 
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These  goals  represent  the  key  performance  parameters  which  serve  as  the 
program  baseline;  this  baseline  is  essentially  a  contract  between  the  program  manager, 
Defense  Acquisition  Executive  and  Service  Acquisition  Executive.  The  approved 
program  baseline  parameters  then  serve  as  control  objectives  in  that  the  program 
manager  must  report  on  the  status  of  approved  baseline  parameters  to  the  acquisition 
executives;  the  status  of  those  baselines  is  also  key  to  a  program’s  migration 
throughout  the  system’s  development. 

Contracts  for  Performance-based  Services:  The  contract  is  the  common 
language  between  the  buyer  and  seller  and,  therefore,  it  should  lend  itself  to 
quantifiable  measures  of  performance.  A  guide  to  best  practices  for  writing 
Performance-Based  Service  Contracts  (P-BSC)  was  published  by  the  Office  of  Federal 
Procurement  Policy  (OFPP).  This  document  applies  the  GPRA  concept  in  its 
instructions  on  the  preparation  of  performance-based  work  statements  (P-BWS).  The 
guide  recommends  describing  the  requirement  in  measurable  performance  standards. 
These  standards  should  include  such  elements  as  “what,  when,  where,  how  many,  and 
how  well”  the  work  is  to  be  performed.  This  requires  defining  the  required  work  in 
quantifiable  terms  with  an  acceptable  error  rate,  which  is  referred  to  as  an  allowable 
quality  level  (AQL). 38 

Aircraft  Service  Contracts:  In  describing  the  value  of  performance  measures  in 
the  management  of  aircraft  and  facility  maintenance  service  contracts,  Ms.  Cheryl  Smith 
of  Fleadquarters  Air  Mobility  Command  (AMC/LGCO)  stated,  “performance  measures 
quantitatively  tell  us  something  important  about  the  contractor’s  products,  services  and 
the  processes  that  produce  them,  i.e. ,  how  well  the  contractor  is  doing,  is  the  contractor 
meeting  its  goals,  are  customers  satisfied,  are  the  contractor’s  processes  in  statistical 
control,  and  if  improvements  are  necessary,  where  are  they  needed?” 39 


38  A  Guide  to  Best  Practices  for  Performance-Based  Service  Contracting,  Final  Edition,  Office  of  Federal 
Procurement  Policy  (OFPP),  Office  of  Management  and  Budget  (OMB),  Executive  Office  of  the 
Government,  October  1998. 


39  Ms.  Cheryl  Smith,  HQ  AMC/LGCO,  “Performance  Measurement,”  Undated,  Briefing,  Scott,  AFB,  IL. 
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AMC  best  practices  include:  (a)  relying  on  commercial-quality  standards,  (b) 
having  the  contractor  propose  metrics  and  a  quality  plan,  (c)  only  measuring  critical 
outcomes  because  excessive  metrics  can  be  costly,  and  (d)  including  language  in  the 
contract  permitting  the  negotiation  of  changes  to  the  measures. 40 

Contractor  Performance  Evaluation  Reporting:  As  noted  earlier,  contractor 
performance  assessments  reports  are  required  and  maintained  in  the  CPARS 
database.  Dollar  thresholds  by  business  sectors  for  reporting  positive  as  well  as 
negative  performance  information  on  DoD  contracts  are  (See  Table  3): 

Business  Sector  Dollar  Threshold 


Systems 

$5,000,000 

Ship  Repair/Overhaul 

$500,000 

Services 

$1,000,000 

Information  Technology 

$1,000,000 

Operations  Support 

$5,000,000 

Fuels 

$100,000 

Table  3.  COPARS  Dollar  Thresholds  by  Business  Sectors 


What  Can  Be  Learned  From  Private  Industry  About  Measuring 
Contractor  Performance? 

Measuring  Supplier  Performance:  From  literature  research,  one  can  conclude 
that  private  industry  collects  supplier-performance  data  primarily  for  strategic  sourcing 
purposes  and  to  determine  how  well  the  purchasing  department  as  a  whole  is 


40  Ibid. 
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performing.  The  primary  purpose  is  to  use  metrics  at  a  macro  level  to  judge 
performance  of  the  procurement  organization  at  the  macro  level. 

Typical  is  a  comment  by  Terry  Sueltman,  Corporate  Vice  President  of  Purchasing 
and  logistics,  Sonoco  Products.  He  believes  purchasing  and  supply-chain  management 
measures,  to  be  effective,  “must  be  more  strategic,  at  a  higher  level  to  have  a  broad 
business  impact  and  should  lead  change.  Instead  of  talking  of  the  price  per  part  we  use, 
talk  about  the  impact  on  production  cost.” 41 

The  Center  for  Advanced  Purchasing  (CAPS)  publishes  a  list  of  “Standard” 
Purchasing  Performance  Measures.  The  significance  placed  on  the  effectiveness  of 
purchasing  organizations  in  a  macro  sense  can  be  judged  by  looking  at  the  standards. 
Examples  of  the  twenty-one  CAPS  standards  are:  purchasing  operating  expense  dollars 
as  a  percent  of  sales  dollars,  percent  change  in  total  purchase  dollars 
influenced/assisted  by  purchasing  and  the  number  of  active  suppliers  accounting  for 
80%  of  purchase  dollars,  percent  of  total  purchase  dollars  spent  through  reverse 
auctions,  and  percent  of  purchase  dollars  spent  through  strategic  alliances. 42 

What  are  the  most  important  supplier  characteristics?  In  February  2002,  The 
Journal  of  Supply  Chain  Management  published  the  results  of  a  random  sampling  of  the 
over  2,000  members  of  the  Institute  for  Supply  Management  (ISM)  members  on  formal 
supplier-evaluation  programs.  An  analysis  of  the  299  responses  was  published  as  a 
report  titled  “Formal  Supplier  Evaluation  Programs  and  the  Factors  Used  to  Channel 
Evaluations.”  A  summary  of  the  responses  showed  the  following. 

•  Less  than  half  of  the  responding  firms  have  formal  supplier  evaluation  processes 
in  place.  (Formal  supplier  evaluations  were  aggregate  evaluations  of  vendor 
performance  rather  than  measuring  ongoing  outputs  of  a  contract  as  part  of 
contractor  surveillance.) 


41  Terry  Sueltman,  “Supplier  Performance  Metrics  That  Really  Make  a  Difference.”  Supplier  Selection  and 
Management  Report  (September  2001 )  1 1 . 

42  “CAPS  ‘Standard’  Purchasing  Performance  Measures:  Supplier  Performance  Metrics  That  Really  Make 
a  Difference”  Supplier  Selection  and  Management  Report.  (September  2001).  12. 
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•  For  those  who  have  a  formal  rating  system, 

■  Two-thirds  ranked  supplier  performance  relative  to  one  another. 

■  Over  half  evaluated  suppliers  on  a  routine  basis. 

■  Twenty-one  percent  of  the  suppliers  were  asked  to  perform  self-evaluations. 

■  Less  than  one-third  of  the  responses  indicated  they  had  a  system  for 
weighing  categories  of  evaluation. 

■  There  were  over  thirty  different  supplier  characteristics. 43  (Table  4  illustrates 
the  Top  Ten  Categories.) 


Top  Ten  Categories 


CATEGORIES/(PERCENTAGE) 

-  Quality  and  Process  Control  (24.9) 

-  Continuous  Improvement  (9.2) 

-  Facility  Environment  (8.2) 

-  Customer  Relationship  (8.2) 

-  Delivery  (8.1) 

-  Inventory  and  Warehousing  (7.0) 

-  Ordering  (5.8) 

-  Financial  Condition  (5.5) 

-  Certifications-  ISO  (3.6) 

-  Price  (3.6) 


RELATIVE 

IMPORTANCE 


1 

2 
2 
2 
2 
2 
2 
2 
3 
3 


No 

Statistical 

Difference 


Penny  M.  Simpson  and  Judy  A  Siguaw,  "Measuring  the  Performance  of  Suppliers:  An  Analysis  of  Evaluation  Processes," 
Journal  of  Supply  Chain  Management:  A  Slob  at  Review  of  Purchasing  and  Supply,  Institute  of  Supply  Management,  February 
2002,  T ables  III  and  IV,  pages  13  -  14 


Table  4.  Top  Ten  Categories 

From  this  survey,  one  can  conclude  that,  for  the  most  part,  commercial  industry 
uses  performance  measurement  primarily  to  evaluate  the  effectiveness  and  efficiency  of 
its  purchasing  department.  The  lack  of  a  formal  supplier-evaluation  program,  like  the 
DoD  CPARS  program,  by  less  than  half  of  those  surveyed  may  be  the  result  of  the  fact 
private  industry  is  not  subject  to  bid-protest  rules  as  is  the  federal  government. 


43  Penny  M.  Simpson,  Judy  A.  Siguaw,  and  Susan  C.  White,  “Measuring  the  Performance  of  Suppliers: 
An  Analysis  of  Evaluation  Processes,  ’’Journal  of  Supply  Chain  Management:  A  Global  Review  of 
Purchasing  and  Supply  Chain  38,  no.  1  (February  2002):  29. 
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How  do  Commercial  Firms  Measure  Services?  An  analysis  published  by  RAND 
on  a  study  of  “fourteen  innovative  commercial  customers  and  world-class  providers  of 
facilities  management  services”  provided  insight  into  how  these  firms  used  performance 
metrics  to  execute  sourcing  activities  and  to  help  manage  their  relationships  and 
promote  continuous  improvement. 

The  report  concluded: 

•  Customer  and  provider  jointly  choose  metrics. 

■  The  customer  tended  to  track  output-oriented  metrics. 

■  The  service  provider  may  also  track  process-oriented  metrics. 

•  Both  qualitative  and  quantitative  measures  were  used. 

•  Both  parties  refined  their  set  of  metrics  as  the  relationship  evolved. 

•  Benchmarking  studies  were  used  to  set  feasible  goals. 

•  Both  formal  and  informal  reviews  were  held  to  communicate  performance. 

•  Both  the  generation  of  metrics  and  tracking  of  performance  are  costly. 

•  Most  common  characteristics  of  metrics  used  are: 

■  Finance  cost  and  customer  satisfaction. 

■  Qualitative  metrics  on  performance  of  specific  services. 

•  Special  interests  topics  were  also  tracked.44 

(See  Table  5:  How  Thirty  Commercial  Firms  Measure  Service.) 


44  Laura  H.  Baldwin,  Frank  Camm,  and  Nancy  Moore,  Strategic  Sourcing:  Measuring  and  Managing 
Performance,  DB-287-AF  (Santa  Monica,  CA:  RAND,  2000). 
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How  Thirty  Commercial  Firms 
Measure  Services 

•  Customer-Provider  Jointly  Choose  Metrics 

-  Both  Qualitative  and  Quantative  Metrics  Used 

•  Customers  Tracked  Output-Oriented  Metrics 

•  Provided  Added  Process  Orientated  Metrics 

-  Facilitates  Communication 

•  Leading  to  More  Responsive  Service 

•  Metrics  Evolved  Over  Time  of  Contract 

•  Bench  Marking  Studies  Used  to  Set  Coals 

•  Added  Benefit 

-  J ustify  Cost  in  Budget 

-  Realign  Workscope 

Laura  H  Baldwin,  Frank  Camm.  and  Nancy  Moore,  Strategic  Sourcing  Measuring  and  Managing  Performance , 
RAND  DB-287-AF,  Santa  Monica,  CA..  2000,  Appendix  (Study  of  Thirty  Customers  and  Providers) 


Table  5.  How  Thirty  Commercial  Firms  Measure  Services 


Conclusion 


Primary  Research  Question:  Can  Performance  Measurement  Be 
Used  to  Manage  a  Contractor’s  Performance?  The  Answer  is 
Yes. 

Performance  measurement  can  be  used  to  manage  contractor's  performance 
after  award. 


The  Government  Performance  and  Results  Act  (GPRA)  of  1993  requires 
managers  in  the  Executive  Branch  to  develop  strategic  plans  and  to  use  performance 
indicators  to  record  output  and  evaluate  the  outcome  of  each  program.  The  Act 
establishes  a  management  framework  that  includes: 

•  Development  of  strategic  performance  program  plans  with  well-defined 
desired  outcomes, 


•  Development  of  performance  Indicators  for  desired  outcomes, 


•  Definitions  of  outputs  to  track  and  measure  results, 
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Comparison  of  outputs  to  desired  outcomes  and, 


•  Documentation  of  report  results.  (Reference  Figure  2.  The  GPRA 
Performance  Accountability  Framework) 

This  framework  readily  lends  itself  to  the  measurement  of  contractor  performance 
and  appears  to  be  an  appropriate  way  to  manage  contracts  in  the  public  sector. 

Currently,  an  integral  part  of  the  contract  administration  (management)  process  is 
contractor-performance-evaluation  reporting.  However,  such  documentation  is  seen 
primarily  as  a  way  to  mitigate  the  risk  of  selecting  a  contractor  with  a  poor  performance 
track  record  in  the  future.  Examples  of  data  that  must  be  reported  on  a  contractor’s 
performance  include:  (a)  Contractor’s  record  of  conforming  to  contract  requirements  and 
standards  of  good  workmanship,  (b)  Contractor’s  record  of  forecasting  and  controlling 
cost,  (c)  Contractor’s  adherence  to  contract  schedules,  including  the  administrative 
aspects  of  performance,  (d)  Contractor’s  history  of  reasonable  and  cooperative  behavior 
and  commitment  to  customer  satisfaction,  and  (e)  Contractor’s  business-like  concern  for 
the  interests  of  the  customer.  It  appears  the  reported  data  could  easily  be  tailored  to  fit 
the  need  of  a  performance-measurement  process. 

This  study  also  notes  Kaplan  and  Norton's  Balanced  Scorecard  concept  enables 
managers  to  visualize  the  breadth  of  an  operation  as  well  as  to  link  measurements  to 
strategy.  As  seen  in  Table  6  below,  a  visual  depiction  of  The  Balanced  Scorecard 
Approach,  the  balanced  scorecard  methodology  could  be  adapted  to  measure 
contractor  performance  in  a  manner  that  would  enable  the  contract  strategy,  as  well  as 
critical  success  factors  and  desired  outcomes,  to  be  included  in  a  contractor- 
performance  measurement  process. 
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A  Balanced  Scorecard  Approach 


CONTRACT  PURPOSE  AND  OBJECTIVES 

Vision 

Critical 

Success 

Factors 

•Admin. 

Cost 

.Cost 

Forecast- 

Control 

•Satisfaction 

•Cost 

•Quality 

•Timeliness 

•Integrity 
•Policy  Coals 

•Business 
Relations 
•Key  Pers. 

•Continuous 

Improve 

Critical 

Measures 

m  FAR  V 
M  Past 

•Cycle 

Time 

•EVS 

ision 

Performance 

•LXT  Results 

•Schedule 

(EVS) 

•Feedback 

(Surveys) 

•Feedback 

(Surveys) 

•Results 

Both 

Table  6.  The  Balanced  Scorecard  Approach 


The  result  would  be  to  add  joint  accountability  for  results  to  the  buyer-seller 
relationship. 

It  is  also  believed  that  both  parties  would  jointly  benefit  from  the  enhancement  of 
communications — communications  that  would  no  doubt  result  in  the  use  of  these 
indicators  throughout  the  life  of  the  contract  rather  than  as  a  reporting  mechanism  after 
the  fact. 
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